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USER CONTROLLED DEVICE FOR SENDING CONTROL SIGNALS TO AN 
ELECTRIC APPLIANCE. IN PARTICULAR USER CONTROLLED POINTING 
DEVICE SUCH AS MOUSE OR JOYSTICK. WITH 3D-M0TI0N DETECTION 

BACKGROUND OF THE INVENTION 

5 Field of the Invention 

The present invention relates to a user-controlled device, in particular 
a mouse or a joystick, with 3D motion detection. The invention is however also 
applicable to game pads, trackballs and other screen pointing devices for a 
computer system as well as to devices for pointing or selecting predetemnined 

1 0 tasks or information according to their position, which are connected to a computer 
or a computer-controlled system. The invention is also applicable to the control of 
an electrical appliance, e.g., for switching on an electrical appliance and activate 
particular tasks, on the base of a 3D movement signal generated by the user- 
controlled device. 

15 Description of the Related Art 

As is known, mice are now the most common interface between a 
person and a computer or a computer controlled device and are hand-displaced on 
a plane or two-dimensional surface to control a cursor or pointer or activate 
particular tasks. To this end, typical mice comprise a plurality of sensors detecting 

20 a 2D movement of the mouse; a plurality of buttons for entering commands and a 
communication interface for communication with the computer system. 

In view of the ease of operation and spread in use of mice as a 
convenient interface with computer systems, a number of functionalities are being 
developed to make mice still easier to use, to reduce operation stresses and 

25 damages to arms and shoulders, to increase the number of tasks that may be 
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controlled or selected through a mouse, to adapt to various specific requirement 
and operation environment or to detect movements with more degrees of freedom. 

For example, a mouse has been proposed, having improved 
movement detection capabilities, including detection of tilting in four different 

5 directions, rotation about its axis and a little vertical movement. This mouse, 
described, e.g., in "The VideoMouse: A Camera-Based Multi-Degree-of-Freedom 
Input Device," by K. Hinckley et al.. ACM UISr99 Symposium on User Interface 
Software & Technology, CHI Letters 1 (1), pp. 103-112, uses a video camera for 
detecting the movement. However, although the image processing systems are 

1 0 becoming cheaper and smaller, the costs and dimensions of these systems do not 
allow their use in all systems. Furthermore, this type of movement detection has a 
functionality highly dependent from light conditions and/or optical features of the 
surface the mouse rests on. 

Furthermore, the known solutions not always allow operation by 

15 disabled persons, having limited or no hand control. 

BRIEF SUMMARY OF THE INVENTION 

An embodiment of the invention improves a user controlled device of 
the indicated type, so as to allow a wider applicability. 

According to various embodiments of the present invention, there is 
20 provided a user controlled device, and a method for generating control signals. 

According to an aspect of the invention, the user controlled device 
accommodates an accelerometer or acceleration sensor made with the MEMS 
(MicroElectroMechanicalSystem) technology and able to detect 3D movements, in 
particular movements around two perpendicular axes, so as to sense the 
25 movement of the user controlled device in the space and send corresponding 
control signals to an electrical appliance, e.g.. a computer system. 

According to a first embodiment, the user controlled device is similar 
to a conventional mouse with buttons, wheels and click possibilities, but instead of 
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being configured so as to be displaceable in a plane, has a support allowing tilting 
of the device around two perpendicular axes. The support may allow a vertical 
displacement of the device body. 

According to another embodiment, the user-controlled device is an 

5 aid device for disabled individuals, In particular for persons having a reduced 
residual mobility and can, e.g.. only move the head. The control device has a 
housing accommodating a dual-axis MEMS accelerometer operating as an 
inclinometer; the housing is wom by the person and fixed to a mobile limb or head. 
For example, the housing may be similar to the housings used for hearing aids. 

1 0 and may be supported in the eyeglass arm. or be fixed to the head through a 
hairband. 

ACCORDING TO ANOTHER ASPECT OF THE INVENTION, THE CONTROL 
DEVICE IS CONNECTED THROUGH A WIRE OR IN A WIRELESS WAY TO THE 
COMPUTER SYSTEM.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE 
15 DRAWINGS 

For the understanding of the present invention, prefen-ed 
embodiments thereof are now described, purely as a non-limitative examples, with 
reference to the enclosed drawings, wherein: 

Figure 1 shows a perspective view of a first embodiment of the 

20 present control device; 

Figure 2 is a lateral view of the pointing device of Figure 1 ; 
Figure 3 shows a perspective view of a second embodiment of the 

present control device; 

Figure 4 shows a perspective view of a third embodiment of the 

25 control device; 

Figure 5 shows a perspective view of a fourth embodiment of the 

control device; 
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Figure 6 shows a perspective view of a fifth embodiment of the 
control device; 

Figure 7 shows a perspective view of a sixth embodiment of the 
control device; 

5 Figure 8 is a block diagram of the pointing device of Figures 1-7; 

Figure 9 is a different block diagram of the control device of Figures 

1-7; 

Figure 10 is another block diagram of the control device of Figures 1- 

7; and 

10 Figure 1 1 is flow-chart of the operation of the pointing device of 

Figure 6 or 7. 

DETAILED DESCRIPTION OF THE INVENTION 

Figure 1 shows a first embodiment of the present control device, In 
the shape of a mouse 1 . The mouse 1 comprises a body 2 of rounded shape 
1 5 having two buttons 3 and a central wheel 4 operable by the user, in a per se known 
manner. In the altemative, a middle button may be provided instead of the wheel. 

The body 2 is supported by a curved base 5 preferably In the shape 
of a spherical cap arranged with the convexity looking downwards, so as to allow 
tilting of the body 2 around two axes A and B, perpendicular to each other and to a 
20 vertical axis C. The curved base 5 may be of rigid or resilient material (e.g., 
rubber), to allow a vertical displacement of the body 2. 

The body 2 accommodates the usual control circuitry (not shown), for 
detecting actuation of the buttons 3 ("clicking") and rotation of the wheel 4 in a per 
se known manner and sending suitable signals to a computer system (see Figures 
25 7-9). Furthermore, the body 2 accommodates a 3D-motion detection device 7 (the 
block diagram whereof is shown in Figures 7-9) based on a MEMS accelerometer 
detecting the acceleration and movement of the body, in particular the tilting of the 
body 2 around axes A and B and in case vertical displacement along the axis C 
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and sending corresponding Information or control signals toward the computer 
system. The transmission circuitry may be in common with the usual button and 

wheel control circuitry. 

In the shown embodiment, an electrical wire 8 connects the mouse 1 
5 with the computer system; in the altemative and in a per se known manner, the 
mouse 1 may be connected wireless (e.g.. by optical or radio transmission) to the 

computer system. 

The mouse 1 is designed to be balanced and to rest in an horizontal 

position in the absence of extemal forces and to tilt around axis B (up-down 

10 movement of the front portion bearing the buttons 3 and the wheel 4) and/or 

around axis A (left-right movement) under the pressure of a user's hand. The 3D- 

motion detection device 7 (as discussed in detail with reference to Figures 7-9) 

detects the tilting and in case the vertical displacement and generates 

corresponding signals to control an arrow on a screen of the computer system. In 

1 5 particular, the left-right mouse movement (around axis A) may cause a 

corresponding left-right movement of an arrow on the screen; the up-down mouse 

movement (around axis B) may cause a corresponding up and down movement on 

the screen; the vertical movement may control further functions, for example for 

handling more folders on the screen or performing other pre-programmed 

20 dedicated tasks. 

Figure 3 shows a different embodiment, wherein the body 2 is 
supported by a suction cap 40 and a spring 41 is interposed between the suction 
cap 40 and the body 2. By virtue of the spring 41 . the body 2 may be tilted around 
axes A and B and be displaced along axis C, and a 3D-motion detection device 

25 (not shown) arranged inside the body 2 sends corresponding control signals to a 
computer system. In the altemative, more springs, e.g.. four springs, may be 
provided for a better tilting control, as schematically represented by dashed lines. 

Figure 4 shows another embodiment, wherein the body 2 is 
supported by four balls 45. e.g.. of rubber. Also here, the body 2 may perfomn a 
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tilting movement around axes A. B and vertical displacement along vertical axis C. 
and a 3D-motion detection device (not shown) arranged inside the body 2 sends 
corresponding control signals to a computer system. 

Figure 5 shows another embodiment, wherein the control device is a 
5 joystick 47, having usual buttons for function control and accommodating a 3D- 

motion detection device 7. 

Figures 6 and 7 show different embodiments specifically intended for 
disabled persons. In this case, a pointing device 10 includes a housing 15 fixedly 
attached to an article worn by the user. 
-10 In Figure 6, the housing 1 5 is attached to a hairband 1 6 and 

accommodates the 3D-motion detection device 7, shown with dashed lines; the 
pointing device may be connected to the computer system through a wire 8 or, 

preferably, in a wireless way. 

In Figure 7, the housing 15 is attached to an arm 18 of a pair of 

15 glasses 19. 

In both cases, tilting of the head of a user up-down and/or left-right 
causes tilting of the pointing device around axes A, B, as visible in the enlarged 
detail of Figure 6, and sending of corresponding signals to the computer system, 
analogously to the embodiments of Figures 1-5. In addition, the pointing device 10 
20 may be designed to associate specific movements of the housing 1 5 to "click" 
functions, as below described in greater detail with reference to the flow-chart of 
Figure 11. 

A first embodiment of the 3D-motlon detection device 7 is shown in 
Figure 8. Here, the 3D-motion detection device 7 includes a 2-axis accelerometer 
25 20 manufactured in the MEMS technology, for example as described in European 
patent application N. 02425320.5 filed on 21.5.2002. Accelerometer 20 generates 
two output signals X, Y proportional to the tilting angle of the mouse 1 or of the 
pointing device 10 around axes A, B. Output signals X, Y are digitized in an 
analog-to-digital converter 21 and fed to a microcontroller 22. Microcontroller 22 
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may also receive further control signals, such as "clicking" signals from the buttons 
3 and position signals from the wheel 4 in the embodiments of Figures 1-5. as 

indicated with dashed line. 

The 3D-motion detection device 7 also includes a communication 
5 port 23. for example, an RS 232 or USB port for signal exchange between the 
microcontroller 22 and a computer 25 including, i.e.. a central unit 26. a screen 27 

and a keyboard 28. 

In case of the mouse 1 or the pointing device 1 0 allow vertical 
displacement, a further accelerometer 29 may be provided to send a 
1 0 corresponding signal to AD converter 21 and microcontroller 22. as shown in 
dashed line. 

Figure 9 shows a second embodiment of the 3D-motion detection 
device 7 comprising, instead of a 2-axis accelerometer, two 1-axis accelerometers 
30 and. in case, a vertical axis accelerometer 29. all manufactured in the MEMS 
1 5 technology. For example, the accelerometer described in U.S. Application No. 
1 0/1 28. 1 33 filed on April 23. 2002 or the accelerometer described in US 5,955.668 
may be used. In this case, the accelerometers 30 are arranged so that one 
accelerometer 30 detects tilting around axis A and the other accelerometer 30 
detects tilting around axis B. while accelerometer 29 detect displacement along 
20 axis C. and generate respective output signals X. Y and Z. 

Figure 10 shows a third embodiment including a 2-axis 
accelerometer 20 as in Figure 8. but communication with the computer 25 is 
accomplished through a radio-frequency link (e.g., using the Bluetooth 
technology). Here, the microcontroller 22 is connected with an RF 
25 transmitter/receiver 31 which, through antennas not shown, communicates with an 
interface 32 including an analogous RF transmitter/receiver 33 and a digital port 
34, e.g.. an RS 232 or USB port, in turn connected with the computer 25. 

In the case of the mouse 1 . the 3D-motion detection device 7 detects 
the tilting movements of the body 2 and controls the position of an arrow or other 
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pointer on the screen 27 on the basis of the output signals X and Y fed by the 2- 
axis accelerometer 20 or the 1-axis accelerometers 30 in a manner analogous to 

conventional mice. 

In case of vertical displacement, signal Z may be used according to 

5 the pre-programmed task. 

In case of the pointing device 1 0. as said, the microcontroller 22 may 
control not only the shifting of an arrow on the screen 27, but also "clicking" 
functions, based, e.g., on the variation speed of the output signals X, Y fed by the 
accelerometer(s) 20, 30. Conveniently, the microcontroller 22 is able to 

10 discriminate among unintentional small movements (e.g.. tremors) of the user's 
head, intentional movements of bigger entity for pointer position control and rapid 

head movements for "clicking." 

To this end. the microcontroller compares the entity of the detected 
movements (difference between the current and previous output signals X and Y, 
15 fed by the accelerometer) with own movement thresholds, to detect an arrow 
control movement, and with a click threshold, greater than the movement 
thresholds, to detect a clicking movement. In practice, the derivative of the signals 
along directions X and Y is calculated and compared with thresholds for controlling 
the position of the arrow on the screen or for controlling clicking functions. 
20 In particular, when an arrow control movement is detected (slow 

movement of the head), the signs of the derivatives indicate the direction (up. 
down. left, right) of the arrow movement with a speed which is a function of the 
amplitude of the derivative. When instead a clicking function is detected, the sign 
of the derivative of one output signal indicates simple clicking of the left or of the 
25 right button, and the sign of the derivative of the other output signal indicates 

continuous pressure of the left button or interruption of the continuous pressure, as 
below described In detail. 
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The thresholds may be programmed by the user in a setup phase of 
the pointing device 10. as well as the functions associated with slow or rapid 
movement. 

A flow-chart of the control program of the microcontroller 22 for the 
5 embodiments of Figure 6, 7 is shown in Figure 11 . 

Initially, step 50. thresholds THC, THX, THY. and constants Kx. Ky 
are initialized. Threshold THC represents the clicking threshold, that is the 
minimum derivative in absolute value for controlling a clicking function; THX 
represents the X-signal threshold, that is the minimum derivative in absolute value 
1 0 for recognizing a valid movement along the X axis and THY represents the Y-signal 
threshold, that is the minimum derivative in absolute value for recognizing a valid 
movement along the Y axis. Kx and Ky represent the desired movement speed. 

Then, the (digitized) output signals X and Y from the 
accelerometer(s) 20, 30 are read, step 52; the entity of the movement in the X 
1 5 direction is calculated as difference between the output signal X and a previous 
value XOLD and compared with positive clicking threshold THC, step 54. If the 
difference X-XOLD is higher than the positive clicking threshold THC, a right click 
(corresponding to clicking of the right button in a conventional mouse) is detected 
and a corresponding signal is sent to the computer system, step 56; otherwise the 
20 difference X-XOLD is compared with the negative clicking threshold -THC, step 58. 
If the difference X-XOLD is lower than the negative clicking threshold -THC. a left 
click (corresponding to clicking of the left button in a conventional mouse) is 
detected and a corresponding signal is sent to the computer system, step 60. 

If the difference X-XOLD is higher than negative clicking threshold 
25 -THC but lower than positive clicking threshold THC, output NO from step 58, the 
absolute value of the difference X-XOLD is compared with X-signal threshold THX 
to discriminate between an Intentional small movement and a control movement, 
step 62. If the absolute value of the difference X-XOLD is higher than X-signal 
threshold THX. a new position X_POS of the mouse on the screen is calculated by 



9 



adding a quantity Kx*X, proportional to the detected output signal X. to the 
previous position OLDX_POS and a corresponding signal is sent to the computer 
system, step 64. 

If the X-direction movement Is lower than the X-signal threshold THX 
5 (output NO from step 62). as well as after detecting a clicking function (after steps 
56, 60) and after calculating the new position X_POS (after step 64), the variation 
of the output signal Y is checked, analogously to what described for the X signal. 
Thus, the entity of the movement in the Y direction is calculated as difference 
between the output signal Y and a previous value YOLD and compared with 
1 0 positive clicking threshold THC, step 66. If the difference Y-YOLD is higher than 
the positive clicking threshold THC, a command analogous to the continuous 
pressure of the left button in a conventional mouse is detected and a 
corresponding signal is sent to the computer system, step 68; otherwise the 
difference Y-YOLD is compared with the negative clicking threshold -THC, step 70. 
1 5 If the difference Y-YOLD is lower than the negative clicking threshold -THC, a 
release command of the left button is detected and a corresponding signal is sent 
to the computer system, step 72. 

If the difference Y-YOLD is higher than negative clicking threshold 
-THC but lower than positive clicking threshold THC, output NO from step 70, the 
20 absolute value of the difference Y-YOLD is compared with Y-signal threshold THY, 
step 74. If the absolute value of the difference Y-YOLD is higher than Y-signal 
threshold THY, a new position Y_POS of the mouse on the screen is calculated by 
adding a quantity Ky*Y proportional to the detected output signal Y, to a previous 
position value OLDY_POS and a corresponding signal is sent to the computer 

25 system, step 76. 

If the Y-direction movement is lower than the Y-signal threshold THY 
(output NO from step 74), as well as after detecting a continuous clicking or 
clicking release function (after steps 68, 72) and after calculating the new position 
Y_POS (after step 76), the previous values XOLD, YOLD, OLDX_POS and 
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OLDY_POS are updated with the current values X, Y, X_POS and Y.POS, step 
78. 

The cycle continues until the pointing device is switched off. 
The advantages of the present invention are clear from the above. In 
5 particular, it is outlined that the detection of a 3D movement by way of an MEMS 
acceierometer causes the control device to be very versatile as regards 
application, features and operativity. In particular, the control device may be 
implemented as a mouse, joystick, trackball, control pad or other control device for 
a screen cursor or for selection among a number of alternatives presented on a 
10 screen or other display. The device may be implemented to allow a simple 

actuation, also by persons having reduced movement capabilities; and additional 
control may be implemented by a same control device. 

Furthermore, the implementation as a mouse requires an actuation 
space smaller than the actual mice, since no planar movement on a resting 
1 5 surface is required. Furthermore, no mouse pad is needed, and the present 
pointing device may be actuated on top of any surface, independently from the 

texture or optical properties thereof. 

The control device with 3D-movement detection by MEMS 

accelerometers manufactured using semiconductor technologies is cheaper than 

20 other prior solutions. 

The device may be used to control actuation of different operations 
or tasks of an electrical appliance, which is very advantageous for disabled 
persons or in case that the user should require the hands free for other activities. 

Finally, it is clear that numerous variations and modifications may be 
25 made to pointing, selection or. generally, control device described and illustrated 
herein, all falling within the scope of the invention as defined in the attached 
claims. 

In particular, the pointing, selection or control device may be 
implemented in any support, such as any mouse, joystick, gamepad. PDA 
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(personal digital assistant, allowing Web surfing, e-mail exchange and so on), 
mobile phone, that is 3D-movable or has a 3D-movable portion. 

Furthermore, the shape of the support allowing tilting of the device 
body may vary; for example, in the embodiment of Figure 4. the yieldable balls 
may be replaced by rigid balls connected to the body 2 through elastic means. 

All of the above U.S. patents, U.S. patent application publications, 
U.S. patent applications, foreign patents, foreign patent applications and non- 
patent publications referred to in this specification and/or listed in the Application 
Data Sheet, are incorporated herein by reference, in their entirety. 
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